
Applicant: 
Serial No.: 
Filed: 
Title: 



Docket No. 1073.060 
HjJif^lTED STATES PATENT AND TRADEMARK OFFICE 

RECEIVED 

SF P 2 4 2001 
TECH CENTER 1600/2900 



Diller et al. 
09/595,096 
06/15/2000 



Docket: 
Group Art Unit: 
Examiner: 



1073.060 
1631 

M. Sheinberg 



MOLECULAR DOCKING TECHNIQUE FOR SCREENING OF 
COMBINATORIAL LIBRARIES 



CERTIFICATE OF MAILING 

I hereby certify that this correspondence is being deposited with the U.S. Postal 
Service as first class mail in an envelope addressed to: Assistant 
Commissioner for Patent and Trademarks, on September 1 7 , 2001 . 



Mary Louis&I Gioeni, Esq. 
Attorney ft^rJApplicant 
Registration No. 41,779 

Date of Signature: 



September 17, 2001 



To: Assistant Commissioner for Patents 
Washington, D.C. 20231 



Declaration Regarding Material Incorporated by Reference 



1 . I, Mary Louise Gioeni, am a practitioner representing the Applicants for the above- 
designated patent application. 

2. The disclosure of the application has been amended to include material incorporated 
by reference. 

3. The amendatory material consists of the same material incorporated by reference in the 
referencing application. 

4. A copy of the relevant pages from the referenced publication is enclosed. 

5. I hereby declare that all statements made herein of my own knowledge are true and 
that all statements made on information and belief are believed to be true; and further that 
these statements were made with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Date: September Q, 2001 



By: 




Mary Louise Gioeni 




:ions 



10.7 Variable Metric Methods i^Multidimensions 



425 



:her-Reeves. 
k-Ribiere. 



xactly zero then ^!/J 



*f re/=dbrent (ax , xx , bx , f Idim , df ldjm , TOL , &xminr> 
toy j<=n; { Construct the vector remits to rejrfrn. 

xi[j] *= xmin; 
pCj] +- xi[j]; 

n 

f ree_vector (xicom, 1 ,n) ; 
free_vector(pcom, l,n)j 



ime thing, but 
id companion 



#include "nruti^h" 

extern int no6m; / Defined in dlinmin. 

extern f loapc *pcom, *xicom, (*nrfuno%^f loat []); 
extern vo*G (*nrdfun) (float [] , A. loat\[] ) ; 



float 
I 



PldimCfloat x) 



ht j; 
/float dffrO.O; 
float *xt,*a' 

xt=vector(l ,nc 
df=vector(l,Brcom) ; 

for (j=l;j^com;j++) N ^[j]=pc>rfmCj]+x*xicom[jJ 
(*nrdfun)Kt,df); 

for (j=yfj<=ncom; j++) df ly^df Cj] *xi corny 
f ree_vdctor (df , 1 , ncom) ; 
f ree /ector (xt , 1 , ncom^ 
retjfrn dfl; 



ac) (float []), 

. n] , moves and 
ion xi from p, 
eturns as fret 
i by calling the 



in); 



fb, 



r CITED REFE5»£NCES AMD FURTHER READING: 

Polak, E. 19rl, Computationaij/ethods in Optimization (New Yo^Academic Press), §2.3. [11 
Jacobs.yOA.H. (ed.) ^977 .Jme^te of the Art in Numerical Ankty§is (London: Acade/rfic 

tfess), Chapter WUyT (by K^Ws^rodlie). [2] 
Sto/r, J., and Bulirsch, R/1 980, lntroduciibi%tg Numerical /fialysis (New YorK§pringe/Veriag), 

§8.7. 



10 J Variable Metric Methods in 
Multidimensions 

The goal of variable metric methods, which are sometimes called quasi-Newton 
methods, is not different from the goal of conjugate gradient methods: to accumulate 
information from successive line minimizations so that N such line minimizations 
lead to the exact minimum of a quadratic form in N dimensions. In that case, the 
method will also be quadratically convergent for more general smooth functions. 

Both variable metric and conjugate gradient methods require that you are able to 
compute your function's gradient, or first partial derivatives, at arbitrary points. The 
variable metric approach differs from the conjugate gradient in the way that it stores 
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and updates the information that is accumulated. Instead of requiring intermediate 
storage on the order of N, the number of dimensions, it requires a matrix of size 
N x N. Generally, for any moderate AT, this is an entirely trivial disadvantage. 

On the other hand, there is not, as far as we know, any overwhelming advantage 
that the variable metric methods hold over the conjugate gradient techniques, except 
perhaps a historical one. Developed somewhat earlier, and more widely propagated, 
the variable metric methods have by now developed a wider constituency of satisfied 
users Likewise, some fancier implementations of variable metric methods (going 
beyond the scope of this book, see below) have been developed to a greater level of 
sophistication on issues like the minimization of roundoff error, handling of special 
conditions, and so on. We tend to use variable metric rather than conjugate gradient, 
but we have no reason to urge this habit on you. 

Variable metric methods come in two main flavors. One is the Dovidon-Fletcher- 
Powell (DFP) algorithm (sometimes referred to as simply Fletcher-Powell). The 
other goes by the name Broyden-Fletcher-Goldfarb-Shanno (BFGS). The BFGS and 
DFP schemes differ only in details of their roundoff error, convergence tolerances, 
and similar "dirty" issues which are outside of our scope [1.2], However, it has 
become generally recognized that, empirically, the BFGS scheme is superior in these 
details. We will implement BFGS in this section. 

As before, we imagine that our arbitrary function /(x) can be locally approx- 
imated by the quadratic form of equation (10.6.1). We don't, however, have any 
information about the values of the quadratic form's parameters A and b, except 
insofar as we can glean such information from our function evaluations and line 
minimizations. 

The basic idea of the variable metric method is to build up, iteratively, a good 
approximation to the inverse Hessian matrix A -1 , that is, to construct a sequence 
of matrices Hj with the property, 

limH i = A- ! (10-7.1) 

i-+oo 

Even better if the limit is achieved after N iterations instead of oo. 

The reason that variable metric methods are sometimes called quasi-Newton 
methods can now be explained. Consider finding a minimum by using Newton's 
method to search for a zero of the gradient of the function. Near the current point 
Xi, we have to second order 

/(x) = /( Xi ) + (x - m) • V/(xi) + |(x - Xt) ■ A ■ (x - X*) (10.7.2) 

so 

V/(x) = V/(xi) + A-(x-Xi) (10.7.3) 

In Newton's method we set V/(x) = 0 to determine the next iteration point: 

x-x^-A^-V/te) (10.7.4) 

The left-hand side is the finite step we need take to get to the exact minimum; the 
right-hand side is known once we have accumulated an accurate H w A 

The "quasi" in quasi-Newton is because we don't use the actual Hessian matrix 
of /, but instead use our current approximation of it. This is often better than 
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using the true Hessian. We can understand this paradoxical result by considering the 
descent directions of / at x^. These are the directions p along which / decreases: 
V/-p < 0. For the Newton direction (10.7.4) to be a descent direction, we must have 



that is, A must be positive definite. In general, far from a minimum, we have no 
guarantee that the Hessian is positive definite. Taking the actual Newton step with 
the real Hessian can move us to points where the function is increasing in value. 
The idea behind quasi-Newton methods is to start with a positive definite, symmetric 
approximation to A (usually the unit matrix) and build up the approximating Hi 's in 
such a way that the matrix Hj remains positive definite and symmetric. Far from 
the minimum, this guarantees that we always move in a downhill direction. Close to 
the minimum, the updating formula approaches the true Hessian and we enjoy the 
quadratic convergence of Newton's method. 

When we are not close enough to the minimum, taking the full Newton step 
p even with a positive definite A need not decrease the function; we may move 
too far for the quadratic approximation to be valid. All we are guaranteed is that 
initially f decreases as we move in the Newton direction. Once again we can use 
the backtracking strategy described in §9.7 to choose a step along the direction of 
the Newton step p, but not necessarily all the way. 

We won't rigorously derive the DFP algorithm for taking Hi into Hi+i; you 
can consult [3] for clear derivations. Following Brodlie (in [2]), we will give the 
following heuristic motivation of the procedure. 

Subtracting equation (10.7.4) at x i+ i from that same equation at x» gives 



where V/j = V/(Xj). Having made the step from to Xj+i, we might reasonably 
want to require that the new approximation H i+ i satisfy (10.7.6) as if it were 
actually A -1 , that is, 



We might also imagine that the updating formula should be of the form H i+ i = 
Hi + correction. 

What "objects" are around out of which to construct a correction term? Most 
notable are the two vectors x» + i - x* and V/i+i - V/*; and there is also Hi. 
There are not infinitely many natural ways of making a matrix out of these objects, 
especially if (10.7.7) must hold! One such way, the DFP updating formula, is 



V/(Xi) • (x - Xi) = -(x - Xi) • A • (x - Xi) < 0 



(10.7.5) 



x <+ i-x i =A- 1 -(V/ i+ i-V/i) 



(10.7.6) 



x i+ i-x i = H i+ i.(V/ i+ i-V/i) 



(10.7.7) 



Hi+i = Hj + 



(x i+1 -Xj)& (x 1+ i -xQ 



1 (x i+ i - Xi) - (V/ i+1 - V/i) 
[Hj ■ (V/, +1 - V/j)] ® [Hj • (V/ t+1 - V/j)] 



(10.7.8) 



(V/ i+ i - V/i) • H, • (V/i +1 - V/i) 



where <g> denotes the "outer" or "direct" product of two vectors, a matrix: The ij 
component of u®v is u l v 3 . (You might want to verify that 10.7.8 does satisfy 10,7.7.) 
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The update formula is exactly the same, but with one additional ten.. 
• • • + [(V/ i+1 - V fi ) • Hi • (V/ i+1 - V/i )] u * u (I0 . 7 9) 



where u is defined as the vector 



u = 



-Xj).(V/ j+1 -VU) 



(10.7.10) 



(You might also verify that this satisfies 10 77 ) 
i» N sttps . if/,,. ,„ai, ac ( ' a7 ' 9) ' d0M f«~ve w to A~' 

can M if you, v J al L S '''' 



•include <math.h> 
♦include "nrutil.h" 
♦define ITMAX 200 
♦define EPS 3.0e-8 
♦define TOLX (4*EPS) 
♦define STPMX 100.0 



Maximum allowed number of iterations 
Machine precision. 
Convergence criterion on x values 
5ca, * d ™ximum step length allowed in 
♦define FREEALL free vector fxi ^ ^ * e Arches. 

free.matrix(hessin f l^nTl) f re 'l™;™*** ; \ 

free.ve C tor(dg,l,n); ' ^-^^^S^^^free.vectorCg, l >n ) ; \ 

void dfpmin(float pD, i n t n fi rta * ~+ , . 

float (*func) (float m J ^T^ 9 lnt * iter ' float *'rat 
Given a starting point pV n ] ( * df ^ [] , float D)) ' 

its gradient as calculated by a routine tHW ti? P erfor ">ed on a function f unc usine 

gradient is input as gtol. « pcTT^T^ °" ^ 

iter (the number of iterations that were oerformJ?! J , * (the location of the minimum), 

mt check,i f its,j- (*func)(float [])) ; 

dg=vector(l,n) ; 
g=vector(l, n ); 
hdg=vector(l,n) ; 

hes8in=matrii(l, n ,l,n) ; 
pnew=vector(l, n ) ; 
xi=vector(l jn ) ; 
fp= s (*func)(p) ; 
(*dfunc) (p (g ) ; 
for (i»l;i<»n;i ++ ) { 

for Cj=l;j<=n;j ++ ) hessin[i] [j]^ 0- 
hes3ia[i][i]-i.o ; J ' 

xi[i] = -g[i] ; 
sum pCi]*p[i] ; 

stpmax=STPMX*FMAX(sqrt(sum) , (float)n) ; 



Calculate starting function value and gra- 
dient, B 

and initialize the inverse Hessian to the 
unit matrix. 

Initial line direction. 



| 

i 



for (its=l;its<-ITMAX;its++) { Main loop over the iterations. 

*iter-its ; 

Insr ch (n , p , f p , g , xi , pneu , f ret , stpmax , ftcheck , f unc) ; 

The new function evaluation occurs in lnsrch; save the function value in fp for the 

next line search. It is usually safe to ignore the value of check. 

fp 31 *fret; 

for (i=l ; i<*=n; i++) { 

xi [i J =pnew [i] -p [i] ; Update the line direction, 

p Ci] =pnew [i] ; ana < the curre nt point. 

test=0 . 0 ; T est f or convergence on Ax. 

for (i=l ; i<-n; i++) { 

temp^f abs (xi [i] ) /FMAX (f abs (p [i] ) , 1 . 0) ; 

if (temp > test) test=temp; 

if (test < TOLX) { 
FREEALL 
return; 

> 

for (i=i;i< sn ;i++) dg[i]-g[i] ; S av e the old gradient, 

(*df unc) (p,g) ; and get t h e new gradient. 

^ eSt ^; „ ^ Test for convergence on zero gradient. 

den=FMAX ( *f ret , 1 . 0) ; 

for (i=l;i<=n;i++) { 

temp=fabs(g[i])*FMAX(fabs(p[i]),1.0)/den; 
if (temp > test) test=temp; 

} 

if (test < gtol) { 
FREEALL 
return; 

} 



for (i-l;i<-n;i+ + ) dg[i>g[i]-dg[i] ; Compute difference of gradients, 
for (i=l;i<=n;i+ + ) { and difference times current matri: 

hdg[i]-0.0; 

for (j»i; j<=n;j++) hdg[i] +=* hessin [i] [j] *dg [j] ; 



} 

fac=fae-sumdg=sumxi=0.0; Calculate dot products for the denomi- 

for (i=l;i<-n;i++) { natorSi 

fac += dg[i]*xi[i] ; 

fae += dg[i]*hdg[i] ; 

sumdg SQR(dg[i]) ; 

sumxi +- SQR(xi [i] ) ; 

> 

if (fac > sqrt(EPS*sumdg*sumxi)) { Skip update if fac not sufficiently posi- 
fac^l.O/fac; t ive. 
fad-l.O/fae; 

The vector that makes BFGS differed from DFP: 
for (i=l;i<=n;i++) dg [i] -f ac*xi [i] -fad*hdg[i] ; 
for (i-l;i<-n;i++) { The BFGS updating formula: 

for (j=i; j<=n; j++) { 

hessin[i][j] += f ac*xi [i] *xi [j] 

-f ad*hdg [i] *hdg [j ] +f ae*dg [i] *dg [ j] ; 

hessin[j] [i] =hessin[i] [j] ; 

} 

} 

for (i=l;i<=n;i++) { Now calculate the next direction to go 

xi[i]=0.0; 6 ' 

for (j = l; j<=n;j++) xi[i] -= hessinfi] [j] *g[j] ; 

^ and go back for another iteration. 

nrerror("too many iterations in dfpmin"); 

FREEALL 
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Quasi-Newton methods like dfpmin work well with the approximate line 
minimization done by lnsrch. The routines powell (§10.5) and frprmn (§10.6), 
however, need more accurate line minimization, which is carried out by the routine 
linmin. 

Advanced Implementations of Variable Metric Methods 

Although rare, it can conceivably happen that roundoff errors cause the matrix H, to 
become nearly singular or non-positive-definite. This can be serious, because the supposed 
search directions might then not lead downhill, and because nearly singular Hi 's tend to give 
subsequent Hi's that are also nearly singular. 

There is a simple fix for this rare problem, the same as was mentioned in §10.4: In case 
of any doubt, you should restart the algorithm at the claimed minimum point, and see if it 
goes anywhere. Simple, but not very elegant. Modern implementations of variable metric 
methods deal with the problem in a more sophisticated way. 

Instead of building up an approximation to A ~~ 1 , it is possible to build up an approximation 
of A itself. Then, instead of calculating the left-hand side of (10.7.4) directly, one solves 
the set of linear equations 

A-(x m -Xi) = -V/(x t ) (10.7.11) 

At first glance this seems like a bad idea, since solving (10.7.1 1) is a process of order 
N — and anyway, how does this help the roundoff problem? The trick is not to store A but 
rather a triangular decomposition of A, its Cholesky decomposition (cf. §2.9). The updating 
formula used for the Cholesky decomposition of A is of order N 2 and can be arranged to 
guarantee that the matrix remains positive definite and nonsingular, even in the presence of 
finite roundoff. This method is due to Gill and Murray [1,2]. 



CITED REFERENCES AND FURTHER READING: 

Dennis, J.E., and Schnabel, R.B. 1 983, Numerical Methods for Unconstrained Optimization and 
Nonlinear Equations (Englewood Cliffs, NJ: Prentice-Hall). [1] 

Jacobs, D.A.H. (ed.) 1977, The State of the Art in Numerical Analysis (London: Academic 
Press), Chapter 111.1, §§3--6 (by K. W. Brodlie). [2] 

Polak, E. 1 971 , Computational Methods in Optimization (New York: Academic Press), pp. 56ff . [3] 

Acton, F.S. 1970, Numerical Methods That Work-, 1990, corrected edition (Washington: Mathe- 
matical Association of America), pp. 467-468. 
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Thus our strategy is quite simple: We always first try the full Newton step, 
because once we are close enough to the solution we will get quadratic convergence. 
However, we check at each iteration that the proposed step reduces /. If not, we 
backtrack along the Newton direction until we have an acceptable step. Because the 
Newton step is a descent direction for /, we are guaranteed to find an acceptable step 
by backtracking. We will discuss the backtracking algorithm in more detail below. 

Note that this method essentially minimizes / by taking Newton steps designed 
to bring F to zero. This is not equivalent to minimizing / directly by taking Newton 
steps designed to bring V/ to zero. While the method can still occasionally fail by 
landing on a local minimum of /, this is quite rare in practice. The routine newt 
below will warn you if this happens. The remedy is to try a new starting point. 

Line Searches and Backtracking 

When we are not close enough to the minimum of /, taking the full Newton step p = 6x 
need not decrease the function; we may move too far for the quadratic approximation to 
be valid. All we are guaranteed is that initially f decreases as we move in the Newton 
direction. So the goal is to move to a new point x new along the direction of the Newton 
step p, but not necessarily all the way: _____ ^ 

f Xnew = Xo ld + Ap, 0 < A < 1 (9.7.6) 

The aim is to find A so that /(xoid + Ap) has decreased" sufficiently. Until the early 1970s, 
standard practice was to choose A so that x new exactly minimizes / in the direction p. 
However, we now know that it is extremely wasteful of function evaluations to do so. A 
better strategy is as follows: Since p is always the Newton direction in our algorithms, we 
first try A = 1, the full Newton step. This will lead to quadratic convergence when x is 
sufficiently close to the solution. However, if /(x new ) does not meet our acceptance criteria, 
we backtrack along the Newton direction, trying a smaller value of A, until we find a suitable 
point. Since the Newton direction is a descent direction, we are guaranteed to decrease / 
for sufficiently small A. 

What should the criterion for accepting a step be? It is not sufficient to require merely 
that /(x new ) < /(x^d). This criterion can fail to converge to a minimum of / in one of 
two ways. First, it is possible to construct a sequence of steps satisfying this criterion with 
/ decreasing too slowly relative to the step lengths. Second, one can have a sequence where 
the step lengths are too small relative to the initial rate of decrease of /. (For examples of 
such sequences, see[1], p. 117.) 

A simple way to fix the first problem is to require the average rate of decrease of / to 
be at least some fraction a of the initial rate of decrease V/ ■ p: 

/(Xnew) < /(Xold) + aV/ ■ (Xnew " Xold) (9.7.7) 

Here the parameter a satisfies 0 < a < 1. We can get away with quite small values of 
a; a = 10~ 4 is a good choice. 

The second problem can be fixed by requiring the rate of decrease of / at x new to be 
greater than some fraction /? of the rate of decrease of / at x 0 i d . In practice, we will not 
need to impose this second constraint because our backtracking algorithm will have a built-in 
cutoff to avoid taking steps that are too small. 

Here is the strategy for a practical backtracking routine: Define 

g{\) = /(xoid +Ap) (9.7.8) 

so that 

g'(X) = Vf-p (9.7.9) 

If we need to backtrack, then we model g with the most current information we have and 
choose A to minimize the model. We start with g(Q) and g'(0) available. The first step is 
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always the Newton step, A = 1. If this step is not acceptable, we have available g(l) as 
well. We can therefore model g(\) as a quadratic: 



9(A) * lg(l) - 3(0) - 9'(0)]A 2 + 9'(0)A + g{0) 
Talcing the derivative of this quadratic, we find that it is a minimum when 

9(0) 



A = 



"2(9(1.) -fl(0) (0)1 



(9.7.10) 



(9.7.11 



Since the Newton step failed, we can show that A ^ i for small a. We need to guard against 

too small a value of A, however. We set A m in — 0.1. 

On second and subsequent backtracks, we model g as a cubic in A, using the previous 
value g(Xi) and the second most recent value g{X 2 ): 

g(X) = aX 3 +bX 2 +g f (0)X + g(Q) (".12) 

Requiring this expression to give the correct values of g at X l and A 2 gives two equations 
that can be solved for the coefficients a and b: 



1 



Xi - A 2 



1/A? -1/Al 1 
-A 2 /A? A1/A2 



<?(Ai)-</(0)Ai-<?(0) 
.<?(A 2 )-</'(0)A2-<?(0) 



The minimum of the cubic (9.7.12) is at 



A = 



3a 



(9.7.13) 



(9.7.14) 



We enforce that A lie between A ma x = 0.5Ai and A min - O.lAi mQvimiim 
The routine has two additional features, a minimum step length alamin and a maximum 
step length stpmax. lnsrcH will also be used in the quasi-Newton minimization routine 
df pmin in the next section. 



#include <math.h> 
#include "nrutil.h" 
#define ALF 1.0e-4 
#define T0LX 1.0e-7 



Ensures sufficient decrease in function value. 
Convergence criterion on Ax. 



void insrchdnt a. float xold[] , float fold, float g[] float p[] , float x[], 

xold where he function f unc has decreased "sufficiently." The new function value .s .returned 
"f rtpmax is an input quantity that limits the length of the steps so that you do not try to 
evaluate the function in regions where it is undefined or subject to overflow, p ,s usua ly the 
Neto ^direc t on The output quantity check is false (0) on a normal exit. It ,s true ) w en 
xTtoo close to xold. In a minimization algorithm, this usually s.gnals "n^nwand can 
be Vnored. However, in a zero-finding algorithm the calling program should check whether the 
convergence is spurious. Some "difficult" problems may require double precs-on ,n th» rout.ne. 
{ 

f loat ' a , alam , alam2 , alamin , b , disc , f 2 , rhs 1 . rhs2 , s lope , sum , temp , 

test .tmplam; 

*check=0; 

for (sum=O.0,i=l;i<=n;i ++ ) sum += p[i]*pLU; 
sum-sqrt (sum) ; 

if ( fT (VS^U-O p[il *= st pm ax/sum ; Scale if attempted step is too big. 

for (slope=0.0,i='l;i< =s n;i++) 

slope += g[i]*p[i] ; , n 
if (slope >« 0.0) nrerrorC'Roundoff problem in lnsrch. J; 
test^O.O; Compute A min . 

for (i=l ; K-n; i++) { 
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temp»f abs (p [i] )/FMAX (f abs (xold [i] ) , 1 . 0) ; 
if (temp > test) test=temp; 

} 

alamin^TOLX/test ; 

alam=1.0; Always try full Newton step first, 

for (;;) { Start of iteration loop. 

for (i-1 ; i<-n; i++) x [i] ^xoldfi] +alam*p[i] ; 

*f=(*func) (x) ; 

if (alam < alamin) { Convergence on Ax. For zero find- 

for (i=l ; i<-n;i++) x [i] =xold [i] ; ing, the calling program should 

*check=l; verify the convergence, 

return; 

} else if (*f <= fold+ALF*alam*slcpe) return; 
else { 

if (alam -= 1.0) 

tmplam = -slope/ (2 . 0* (*f -fold-slope) ) ; 

else { 

rhsl = *f-f old-alam*slope; 
rhs2=f 2-f old-alam2*slope ; 

a=(rhsl/(alam*alam)-rhs2/(alam2*alam2))/(alam-alam2) ; 

b= (-alam2*rhs 1/ (alam*alam)+alam*rhs2/ (alam2*alam2) ) / (alam-alam2) 

if (a == 0.0) tmplam = -slope/ (2 .0*b) ; 

else { 

disc-b*b-3 .0*a*slope ; 
if (disc < 0.0) tmplam=0. 5*alam; 
else if (b <= 0.0) tmplam=(-b+sqrt (disc) )/(3.0*a) ; 
else tmplam--slope/(b+sqrt (disc) ) ; 



Sufficient function decrease. 
Backtrack. 

First time. 

Subsequent backtracks. 



> 

if (tmplam > 0.5*ala 
tmplam=0 . 5*alam ; 

} 

} 

alam2=alam; 
f2 = *f; 

alam 3! FMAX(tmplam,0. l*alam) ; 



a) 



A < 0.5\ v 



X > O.lAi. 
Try again. 



Hare now is the globally convergent Newtfcn routine newt thaLuses lnsrch. \ feature 
of newy u that yojfheednpt supS/ the Jacobianxriatrix analytically; th\routinjTwill attempt to 
compuleme necessary parHal derivatives of F^yNfinite differences in th\rouftne f dja£vhis 
Dutinfc uses sorrfc of the techVques describecMn §5S? for computing numeVjal derivatives. ^Qf 
3ursi, youVan/always replace^djac with /routin\that /alculates the Ja&bian anafrticalty 
itt th/s is easV/for you to yao. 

#iW;lude <mdtk.h> 
#i][clude "n/u^il.h 
#oWine MAIITsVtO 
#ieVine T0LF l.o\-4 

leMne TtlLMIN 1 
|fdef\ne TfiLX l.Oe 
frdefifae SpMX 100.0] 
/Here WftXITS is the nja: 
[ function^ values; TOLfllN 
minimurnVof f min ha: 
maximunAstep lengt 



num numfier of iter\tioi 
its the criterion for 
occuiVed; THLX is the con\ 
alloweo\in ine searches. 



T0LF sets tfre Convergence 
iding whether/ spurVous com 
rgence criterio/i on (5x\STPM/ 



criterion on 
rgence to a 
is the scaled 



int nn; 
float *fvec 
void (*prf unVv) (i| 
#def ine? FREEBETUF 



Slobal va/iables to communicate with f min. 



it n, floa 
{f ree.ve 



(] , float 



torVfvec, l,n) ; f ree^veoiir (xold , 1 ,n) ; 



f reb_vectoV(pl 1 ,n) ; freel vect\r (g, l f n) 
f refe_ivecto\(tndx, 1 ,n) ;return\} 



□ ); 



; free_matr\Lx(f jac , l,n t l,n) ;\ 



